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[57] ABSTRACT 

Apparatus is described for processing a digitized image 
signal obtained from an image sensor having color photo- 
sites aligned in rows and columns that generate at least three 
separate color values but only one color value for each 
photosite location, and a structure for interpolating color 
values for each photosite location so that it has three 
different color values. The apparatus includes a memory for 
storing the digitized image signal and a processor operative 
with the memoiy for generating an appropriate color value 
missing from a photosite location by the inteqx>lation of an 
additional color value for such photosite locations from 
color values of different colors than the missing color value 
at nearby photosite locations. The processor also includes 
structure for obtaining Laplactan second-order values, gra- 
dient values aod color difference bias values in at least two 
image directions from nearby photosltes of the same column 
and row and for adding the Lapladan second-order values, 
gradient values and color diffo-ence bias values to define a 
classifier and for selecting a preferred orientation for the 
intopolation of the missing color value based upon a 
dasrifier. Finally, a arrangement is provided for inteipolat- 
ing the missing color value from nearby muMple color 
values selected to agree with the preferred orientation. 

19 Qaims, 6 Drawhig Sheets 
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ADAPTIVE COLOR PLANE 
INTERPOLATION IN SINGLE SENSOR 
COLOR ELECTRONIC CAMERA 

CROSS REFERENCE TO RELATED 
APPLICAnONS 

Reference is made to commonly assigned U.S. Scl Nos. 
08/407,436 filed Mar. 17, 1995 (U.S. Pat No. 5,506,619) to 
James E. Adams, Jr. and John R Hamilton, Jr. and 08/407, 
423 filed Mar. 17, 1995 to John R Hamilton, Jr. and James 
E Adams, Jr. filed concurrently herewith, the teachings of 
which are incorporated herein. 

1. Field of the Invention 

This invention relates to the fidd of electronic imaging 
and is particularly suitable to electronic still imaging by use 
of an electronic still camera having a single color sensor and 
memory. 

2. BACKGROUND OF THE INVENTION 

Id electronic color imaging, it is desirable to simulta- 
neously capture image data in three color planes, usually 
red, green and blue. When the three cdor planes are 
combined, it is possible to create high-quality color images. 
Capturing these three sets of image data can be done in a 
number of ways. In electronic photography, this is some- 
times acconqilished by using a single two dimensional array 
of sensors that are covered by a pattern of red, green and 
blue filt^, This type of sensor is known as a color filter 
array or CFA Below is shown the red (R), green (G) and 
blue (B) pixels as are commonly arranged on a CFA sensor. 

When a color image is captured using a CFA, it is 
necessary to interpolate the red, green and blue values so 
that tiiere is an estimate of all three color values for each 
sensor location. Once tiie iDtcipolation is done, each picture 
element, or pixel, has three color values and can be pro- 
cessed by a variety of known image processing techniques 
depending on the needs of the system. Some exaniples of the 
reasons for processing are to do image sharpening, color 
correction or halftoning. 

The following shows how red green and blue pixels can 
be arranged in a particular color filter array pattern, herein- 
after refrared to as the Bayer color filter array. For a more 
detailed description see U.S. Pat No. 3,971,065 to Bayer. 
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SUMMARY OF THE INVENTION 

The object of this invention is to provide an improved 
apparatus for estimating the missing pixel values in a CFA. 

This object is achieved in apparatus for processing a 
digitized image signal obtained from an image sensor having 
color photositcs aligned in rows and columns that generate 
at least three separate color values but only one color value 
for each photositc location, means for interpolating color 
values for each photosite location so that it has three 
different color values comprising: 
means for storing the digitized image signal; 
a processor operative with said storing means for gener- 
ating an appropriate color value missing from a pho- 
tosite location by the interpolation of an additional 
color value for such photosite locations firom color 



values of different colors than the missing color value 
at nearby photosite locations, said processor including 

means for obtaining Laplacian second-order values, color 
difference bias values, and gradient values in at least 
5 two image directions firom nearby photosites of the 
same column and row; 

means for adding the Laplacian second-order values, the 
color difference bias values, and the gradient values to 
define a classifier and for selecting a preferred orien- 
tation for the interpolation of the missing color value 
based upon a classifier, and 

means for interpolating the missing color value from 
nearby multiple color values selected to agree with the 
preferred orientation. 
Advantages 

The advantages of this invention are 1) is con^^utationally 
efficient both in execution time and memory storage require- 
ments; and 2) by use of the combination of the Laplacian 
2Q second-order values, gradient values and cdor difference 
bias values to produce a classifier, artifacts (color 
inteipolatioD) in output image are substantially reduced 

BRIEF DESCRIPTION OF THE DRAWINGS 

25 FIG. 1 is a block diagram of an electronic still camera 
employing interpolation processing according to the inven- 
tion; 

FIG. 2 is a block diagram of the logic of the interpolation 
processing technique used in connection with the invention; 
^ FIG. 3 is a detailed block diagram of the logic of flie 
interpolation luminance section in FIG. 2; 

FIG. 4 is a more detailed block diagram of the logic of fiie 
chrominance section in FIG. 2; and 
35 FIGS. 5 and 6 show logic block diagrams for the inter- 
polation processes in accordance with the invention. 

DETAILED DESCRIFIION OF THE 
INVENTION 

40 Since single-sensor electronic cameras employing color 
filter arrays are well known, the present description will be 
directed in particular to elements forming part of, or coci>- 
erating more directly with, apparatus and method in accor- 
dance with the present invention. Elements not specdficaUy 

45 shown or described herein may be selected from those 
known in the art 

Referring initially to FIGS. 1 and 2, an electronic still 
camera is divided generally into an input section 2 and an 
inteipdarion and recording section 4. The input section 2 

50 includes an exposure section 10 for directing image lig^t 
from a subject (not shown) toward an image sensor 12. 
Although not shown, the exposure section 10 includes 
conventional optics for directing the image light through a 
dij^jhragm, which regulates the optical aperture, and a 

55 shutter, which regulates exposure time. The image sensor 12, 
which includes a two-dimensional array of photosites cor- 
responding to picture elements of the image, is a conven- 
tional charge-coupled device (CCD) using either well- 
known interline transfer or fraine transfer techniques. The 

60 image sensor 12 is covered by a color filter array (CFA) 13, 
known as the Bayer array, which is described in U.S. Pat 
No. 3,971,065 and herewith incorporated by reference. In 
the Bayer geometry each color covers a photosite, or picture 
element (pixel), of the sensor. In particular, chrominance 

65 colors (red and blue) are interspersed among a checkerboard 
pattern of luminance colors (green). The image sensor 12 is 
exposed to image light so that analog image charge infor- 
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mution is generated in respective photosites. The charge "stack ap*" at video rates. The size of the buffer is estabJished 

information is ^lied to an ou^ut diode 14, which converts to hold enough consecutive images to cover most picture- 

the charge Infonnatlon to analog image signals conespond- taking situations. 

ing to respective picture elements. The analog image signals An operation display panel 30 is connected to the control 

arc s^lied to an A/D converter 16, which generates a digital 5 processor 20 for displaying informatloQ useful in operation 

image signal from the analog input signal for each picture of the camera. Such information might include typical 

element. The digital signals are applied to an image buffer photogr^hic data, such as shutter speed, aperture, exposure 

18, which may be a random access memory (RAM) with bias, color balance (auto, tungsten, fluorescent, daylight), 

storage c^adty for a plurality of still images. field/frame, low battery, low light, exposure modes (aperture 

A oontrolprocessoT 20 generally controls die ii5)ut section preferred, shutter preferred), and so on. Moreover, other 

2 of flie camera by initiating and controlling exposure (by information unique to this type of camera is displayed. F6r 

operation by the diaphragm and shutter (not shown) in the instance, the removable memory card 24 would ordinarily 

exposure section 10), by generating the horizontal and include a directory signifying the beginning and ending of 

vertical clocks needed for driving the image sensor 12 and eadi stored image. This would show on the display panel 30 

for docking image infonnarion therefrom, and by enabling ^5 as dther (or both) the number of images stored or the 

the A/D converter 16 in conjunction with the image buffer 18 number of image spaces remaining, or estimated to be 

for each signal segment relating to a picture element. (The remaining. 

control processor 20 would cffdinarily indude a miaopro- The digital signal processor 22 interpolates each still 

ccssor coupled with a system timing circuit.) Once a certain video image stored in the image buffer 18 according to the 

number of digital image signals have been accumulated in interpolation technique shown in FIG. 2. The interpdation 

the image buffer 18, the stored signals are applied to a digital of missing data vakies at each pixel location follows the 

signal processor 22, which controls the throughput process- sequence shown in FIG. 2; that is, first, the high frequency 

ing rate for the interpolation and recording section 4 of the information for the **missing green" pixds (i.e., the red and 

camera. TTie digital signal processor 22 applies an intcrpo- blue pixel locations) are interpolated to irD|>rove the lumi- 

lation algorithm to the digital image signals, and sends the 25 nance rendition and, secondly, the color difference informa- 

inteipolatcd signals to a conventional, removable memory tion is interpolated at the higji frequency locations by 

card 24 via a connector 26. bilinear methods to generate the other colors of the CFA 

Since the interpolation and related processing ordinarily pattern. In the implementation shown in HG. 2, an adaptive 

occurs over several steps, the intermediate products of the interpolation technique is used in the luminance section 36 

processing algorithm are stored in a processing buffer 28. 30 for optimizing the p^ormance of the system for images 

(The processing buffer 28 may also be configured as part of with horizontal and vertical edges. **Missing green" pixels 

the memory space of the image buffer 18.) The number of are ad^tively interpolated either horizontally, vertically or 

image signals needed in the image buffer 18 before digital two-dimensionally depending upon the gradients, Lapladan 

processing can begin depends on the type of processing, ttiat second-order values and color difference bias values estab- 

is, for a ndghb<ffhood interpolation to begin, a block of 35 lished from the neighboring pixel locations in the vertical 

signals induding at least a portion of the image signals and horizontal directions around the ''missing green" pixel 

comprising a video frame must be available. Consequently, The first step for adaptively interpolating the "missing 

in most circumstances, the interpdation may commence as green" pixels is to select an intctpolation method. The 

soon as the requisite block of picture elements is present in details of this process are shown in block 40 of FIG. 3. The 

the buffer 18. 40 process starts by computing two coni^osite pixel dassificr 

The input section 2 operates at a rate commensurate witii values in block 50, one for the horizontal direction and one 

normal operation of the camera while interpolation, which for the verticaL The term '^pixel classifier*' denotes a value 

may consume more time, can be relatively divorced from the coniputed for the purpose of making a dedsion about further 

input rate. The exposure section 10 exposes the image sensor processing of the pixd information. The tcim "conqjositc" 

12 to image light for a time period dependent upon exposure 45 denotes the dependency of the value on a multq)]idty of 

requirements, for exanq)lc, a time period between 1/1000 color planes. In this case, twice the absolute value of the 

second and several seconds. The image charge is then swept gradient of the green plane is added to the absolute value of 

from the photosites in the image sensor 12, converted to a the Lapladan second-order value of the red or blue plane, 

digital format, and written into the image buff^ 18, The depending on which was the corresponding color in the 

driving signals provided by the control processor 20 to the 50 Bayer color fUter array, and tiie absolute value of the color 

image sensor 12, the A/D converter 16 and the buffer 18 are difference bias value, which uses both green plane and, 

accordingly genaated to achieve such a transfer. The pro- depending on which was the ccrrcsponding color in the 

cessing throughput rate of the interpolation and recording Bayer color filter array, red or blue plane values, 

section 4 is determined by the speed of the digital signal The two dassifier values are compared in block 51 versus 

processor 22. 55 a predetermined value herehiafter referred to as **the first 

One desfrable consequence of this architecture is that the threshold value". If both dassifier values are less than the 

processing algorithm employed in the interpolation and first threshold value, then the inteqwlation method corre- 

recording section may be sdected for quality treatment of sponding to slowly changing scene detail is selected in block 

the image rather than for throughput speed. This, of course, 52. "Slowly changing scene detail" means both horizontal 

can put a delay between consecutive pictures which may 60 direction classifier value and vertical direction dassifier 

affect the user, dq)ending on the time between photogr^hic value are less than the first threshold value, ff at least one of 

events. This is a problem since it is well known and the dassifier values is greater than or equal to the first 

understood in the fidd of dectronic imaging that a digital threshold value, both classifier values are compared in block 

still camera should provide a continuous shooting c^ability 53 to a predetermined value hereinafter referred to as *1he 

for a successive sequence of images. For this reason, the 65 second threshdd value". If both classifier values are greater 

image buffer 18 shown in FIG. 1 provides for storage of a than the second threshdd value, then the interpdation 

plurality of images, in effect allowing a series of images to method corresponding to rapidly changing scene detail is 
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selected in block 54. **R^idly changing scene detail" means 
both horizontal direction classifier value and vertical direc- 
tion classifier value are greater than the second threshold 
value. If at least one of the dassificr values is less than or 
equal to the second threshold value, then the classifier values 
are compared in block 55 to each other. In the likdy event 
that one of the classifier values is less than the other 
dassificr value, the interpolation method corre^onding to 
the smaller classifier value in block 57 is chosen. If both 
dassificr values are equal in value, then the default inter- 
polation method in block 56 is chosen. At this point block 40 
is con^jlete. 

The green Qvaaa) interpolation step in block 44 has two 
parts, as shown in FIG. 4. The first part in block 80 averages 
the two luminance (green) values according to the sdccted 
interpolation method. The second part in block 82 adds a 
correction factor based on dther red or blue neighboring 
values depending on if ihc pixd in question was covered by 
a red or blue filter in the Bayer color filter array and the 
intcipolatioa method selected by block 40, 

The red/blue (chroma) interpolation proceeds in a manner 
similar to the green (luma) interpolation described above. 
The details of this process are shown in Block 46 of FIG. 5. 
The process starts by determining if the two horizontally 
adjacent pixels to the pixel with the '"missing" chroma value 
in question (either red or blue) are of the same color as the 
missing chroma value in block 60. If the two horizontally 
adjacent pixel values are the correct color, then the horizon- 
tal interpolation method in block 61 is used. If the two 
vertically adjacent pixds are of the same colca: as the 
**missing*' chroma value in block 62, then (he vertical 
interpolation method in block 63 is chosen. If ndthcr 
horizontal or vertical intapolation methods have been 
chosen, then compute two canq)osite pixd classifier values 
in block 64, one for the negative diagonal direction and one 
for the positive diagonal. The term "negative diagonal" 
denotes tiie line of slope -1 (connecting the upper left to the 
lower right). The term "positive diagonal" denotes the line 
of slope +1 (connecting the lower left to the upper right), 
^ain, these composite classifiers are found by adding the 
absolute value of the Lapladan second-order value of the 
green plane to the absolute value of the gradient of cither the 
red or blue plane, depending on which color is bdng 
interpolated. 

The two dassifier values are then con^ared in block 65 
and tested for equality. In the likely event that one value is 
smaller than the otho:, the interpolation method cosiespond- 
ing to the smaller vahie is sdected in block 67. If the values 
are equal, then the default hiterpolation method is sdected 
in block 66. At this point Block 46 is done. 

The red/blue (chroma) intapolation step in block 48 has 
two parts, as shown in FIG. 6. In die first part in block 70 
two chrominance values, either red or blue depending on the 
pixel's position in the Bayer color filter airay, are averaged 
according to the selected interpolation radhod. The second 
part in block 72 adds a correction factor based on green 
neighboring values. 

More specifically, the following is a detailed description 
of the operation of the digital signal processor for a specific 
example using the Bayer array. 

Green Plane Interpolation 

The first pass of the interpolation fully populates the green 
color plane. The Bayer color filter array is assumed. Con- 
sider the following pixd neighborhood. 
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Gx is a green pixel and Ax is either a red pixd or a blue 
pixel (All Ax pixels will be the same color for the entire 
neighborhood.) For simplidty, we wiU use the term 
"chroma" to mean dther red or blue. Wc form the following 
classifiers. 

A=M5f2A5-A7h-2IG4-G6W-G4+2(A5tAA>-06l 
^5 v=I-Al+2A5-A9k-2-G2H78h-Ka+2(A5+^n')-<78l 

where 

fcHfrft'+A3-G3y2 

where bh' is the previous value of bh along the same row, bv' 
is the previous value of bv along the same column and Gl' 
and G3' are previously predicted green values. Both bh and 
bv are initialized to zero at the begiiming of therow/cdumn. 

25 (Note that as the algorithm is usually in^lemented to 
process the image row by row, bh can be a simple variable 
and bv is.a one-dimensional array with an entry for each 
column). Note that Ax is either Rx or Bx. (All Ax's will be 
the same color). 

30 These classifiers are con^sed of Laplacian second-OTder 
values for the chroma data, gradient values for the green data 
and color diff^^ence bias values that use both chroma and 
green data. For example, for the classifier h, -AJf 2A5-A7 
is a Lapladan second-order value, G4-G6 is a green gradi- 

35 ent value and -G4+2(A5+4)h>-<j6 is a color difference bias 
value. As such, these dassifiers are sensing the high spatial 
frequency information present in the pixel neighboriiood in 
the horizontal (h) and vertical (v) directions. 
We then form four predictors. 

40 

\fe(GiK»y2+<-Al+2A5-A9y4 

These predictors are composed of arithmetic averages for 
the, green data and. In the case of F, H and V, appropriatdy 
scaled Laplacian second-order values for the chroma data. 

50 Predictor A is to be used when both horizontal and vertical 
dassifier values are less than the first threshold value, Tl. Tl 
is chosen to reduce noise in the image reconstruction in 
regions of slowly changing scene detail. An appropriate 
value for Tl is 25 when the green pixel values and chroma 

55 pixd values range from 0 to 255. Predictor F is used when 
both horizontal and vertical classifier values are greater than 
the second threshold value, T2. T2 is chosen to reduce noise 
in the image reconstruction in regions of rf^dly changing 
scene detail An appropriate value for T2 is 200 when the 

60 green pixd values and chroma pixel values range from 0 to 
255. Predictor H is to be used when the preferred orientation 
for the interpolation is in flie horizontal direction within the 
pixel ndghborhood, Similarly, predictor V is to be used 
when the preferred orientation for the interpolation is the 

65 vertical dh-ection. 

The complete green interpolation process may now be 
expressed as below. 
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is blue and visa versa). Note that all Gbc pixels, Gl through 

if (A < 71 and v< 71) G9, are assumed to be known and coincident with all 

, ^^.7^^ . corresponding Ax and C5 pixels. 

05 -F There are three cases. Case 1 is when the nearest same 

else if (A <v) 5 colored neighbors to Ax are in ^e same column. The 

GS=H following predictor is used. (A4 is used as an example.) 

else 

The key to this process is that both the green and die Case 2 is when the nearest same colored neighbors to Ax 

chroma data must indicate a mimmum of high spatial to ^n. * « • j'-* • j /a^* • 

, . r, ^ j: ■ * * w u are m the same row. The foUowms predictor is used, (A2 is 

frequency infonnation for a given orientation to be chosen . . . ^ ^ 

as the preferred orientation for the inteipolation. If there is ^ examp e.) 

a large mount of high spatial frequency information in either A2=<Ai4:A3y24<-Gi+2C2-C3y2 

the green data or chroma data for a given orientation, it will 

inflate the value of the cciresponding classifier. This, in turn, 1 5 Case 3 is when the nearest same colored ndghbors to Ax 

reduces the likelihood for that orientation to be chosen as the are at the four comers. The following predictor is used. (AS 

preferred orientation for the interpolation. is used as an example.) 
Green Pixel Value Averaging 

Ater the green plane reconstruction, additional smoothing A5^AUA^A7+A9y4^'Gi-aMGS^-09y4 

"f^"^ ""^n^ be p^onncd to 20 improvement for case 3. We form the 

promote further noiscrcduction. Consider the foUowmg post f^^^^^j^^ ^wo classitos 

green plane reconstruction ndghboihood: ^ 

DN=={-Gi+2GS^mi-A9\ 



G2' G3 25 DP=hG3+2jGS-a7MA3'A7\ 
GS Ge 

^ ^ These classifiers are composed of Laplacian second-order 

values for the green data and gradient values for the chroma 

where Gx is an cnginai green pixel value and Gx' is a data. As such, these classifiers are sensing the high spatial 

predicted green pixel value. If two or more of the Gx' values 30 fircquency information present in the pixel neighborhood in 

were interpolated using the method coiresponding to slowly the negative diagonal (DN) and positive diagonal (DP) 

changing scene detail, then Gx is replaced with the follow- directions, 

ing value, G5": We then form three predictors. 

GS'^GX^'+06'+GSry4 35 ASI^Al-yS3^-Gi'^2GS-C9)l2 

One embodiment of this idea is to use G5" when two or more ASi^A3¥ATy2M-GMGS-07y2 

values of Gx' were predicted using predictor A. This need ASA^Al^AUAl^A9y4H^~GMG5^^y4 

not be, however, as any method of mtecpolahon apprqmatc 

to slowly changing scene detail could be used in place of These predictors are composed of arithmetic averages for 
predictor A. (Note that if all four predicted values wmc ^ the chroma data and appropriately scaled L^lacian second- 
derived using predictor A, then G5- becomes, cquivalently: order values for the green data. A5N is to be used when the 
_ preferred orientation for the interpolation is in the negative 
G7MC7i+2c^2GW544G7^2GiOf 2Gm<a3yi6 ^^^^^ ^^^^^^ ^^^j neighborhood. Similarly, 

where the indices refer to fte expanded pixel neighborhood 45 ^ K^. be used when the preferred orientation for the 
1^2^^. interpolation is the positive diagonal direction. ASA is used 

when there is no clear preference for orientation for the 
Qi interpolation. 
02 G3' 04 The complete case 3 chrome interpolation process may 

50 ^® ejqjressed as below. 



010 Gir 012 

Gi3 



i£{pN<DF) 
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In effect, G7" is a center weighted average.) ""^as^^ASP^ 

Red and Blue (Chroma) Interpolation ^j^^ 

The second pass of the interpolation fully populates the AS = ASA 
red and blue color planes. In U.S. Pat No. 4,642,678, issued 

Feb. 10, 1987, Cok (flie disclosure of which is incorporated ^ process both the green and the chroma data must 

by reference hcrcm) disclosed the chroma interpolation indicate a minimmn of high spatial frequency hiformatlon 

summarized below. Consider the following ncighboihood. * g^^en orientation to be chosen as the preferred orien- 

60 tation for the interpolation. If there is a large amount of high 

' spatial frequency infonnation in either the green data or 

Ai G2 A3 chroma data for a given orientation, it will inflate the value 

of the corresponding classifier value. This» in turn, reduces 

the likelihood for that orientation to be chosen as the 

63 preferred orientation for the interpolation. 

Gx is a green pixel, Ax is either a red or blue pixel and C5 In practice, the case 3 diroma interpolation process may 

is the opposite color pixel to Ax (ie., if Ax is red then C5 be simpUfied, as below, for computational efficiency. 
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i£{DN<^DP) 
AS^ASN 

else 

This simplification is achieved by deiiiuiig the negative 
diagonal direction as the default preferred orientation for tiie 
inteipolatioD when both negative and positive diagonal 
dassifiexs are equal. The numb^ of occurences in a typical 
image >^en the negative and positive diagonal classifiers 
are equal is so small that this simpliiicatioD generally has 
negligible impact on the image quality of the final recon- 
structed image. 

The invention has been described in detail with particular 
reference to certain preferred embodiments thereof, but it 
will be understood that variations and modifications can be 
effected within the spirit and scope of the invention. 
PARTS LIST 
2 input section 
4 recording section 
10 exposure section 

12 image sensor 

13 color filter array 

14 ou^ut diode 
16 A/D converter 
18 image buffer 

20 control processor 

22 digital signal processor 

24 removable memory card 

26 connector 

28 processing buffer 

30 display panel 

36 luminance section 

38 chroma section 

40 select best luma intapolation 

44 Interpolate missing luma values 

46 select best chroma interpolation 

48 interpolate missing chroma values 

50 compute horizontal and vertical composite classifier 
values 

51 classifier tiireshold #1 test 

52 select slowly changing scene detail method of interpo- 
lation 

53 classifier threshold #2 test 

54 select rq)idly changing scene detail method of interpo- 
lation 

55 classifier equality test 

56 select default method for interpolation 

57 select the interpolation method corresponding to the 
smaller value 

60 horizontal neighbcaing pixd test 

61 select horizontal interpolation method 

62 vertical neighboring pixel test 

63 select vertical interpolation method 

64 compute positive and negative composite classifier val- 
ues 

65 classifier equality test 

66 select default method for interpolation 

67 select interpolation method caircsponding to the smaller 
value 

70 average red/blue values 
72 add green correction factor 
80 average green value 
82 add red/blue cosrection factors 
We claim: 

1. In apparatus for processing a digitized image signal 
obtained from an image sensor having color photosites 
aligned in rows and columns the image sensor generating at 



least three separate color values but only one color value fa- 
each photosite location, the apparatus including means for 
interpolating the color values for each photosite location so 
that each photosite has the tiiree separate color values 
comprising: 
means for storing the digitized image signal; 
processor means operative with said storing means for 
generating an appropriate color value missing from a 
photosite location by tiie interpolation of an additional 
color value for such photosite locations firom colar 
values of different oolors than the missing color value 
at nearby f^otosite locations, said processor including 
means for obtaining Lapladan second-order values, color 
difference bias values, and gradient values in at least 
two image directions fix)m nearby photosites of the 
same colunm and row; 
means for adding the Lapladan second-order values, the 
color difference bias values, and the gradient values to 
define a classifier and for selecting a prefezred orien- 
tation for the interpolation of the missing color value 
based upon the classifier, and 
means for interpolating the missing color value from 
nearby multiple color values selected to agree with the 
preferred orientation. 

2. The apparatus of daim 1 wherein, when the preferred 
orientation is horizontal, a color value is red, and tiie missing 
color value is green, adjacent horizontal green color values 
are averaged and horizontal red color values including the 
red color value provide a correction value. 

3. Tlie ^paratus of daim 1 wherdn, when the preferred 
orientation is horizontal, a color value is blue, and the 
missing color value is green, adjacent horizontal green color 
values are averaged and horizontal blue color values indud- 
ing the blue color value provide a correction value. 

4. The apparatus of daim 1 v^erein, the classifier indude 
a horizontal dassifier value and a vertical classifier value 
and when the horizontal dassifier value and the vertical 
classifier value are both less than a predetermined value, and 
the missing color value is green, adjacent horizontal and 

40 vertical green color values are averaged to provide a cor- 
rection value. 

5. The apparams of daim 1 wherein, the dassifier indude 
a horizontal dassifier value and a vertical dassifier value 
and when the horizontal dassifier value and the vertical 
classifier value are both greater than a predetermined value, 
a color site is red, and the missing color value is green, 
adjacent horizontal and vertical green color values are 
averaged and horizontal and vertical red color values indud- 
ing a red cdcr site value provide a correction value. 

6. The aj^wratus of daim 1 wherein, the dassifier indude 
a horizontal dassifier value and a vertical dassifier value 
and when the horizontal dassifier value and the vertical 
dassifier vahie are both greater than a predetermined value, 
a colcr site is blue, and the missing color value is green, 
adjacent horizontal and vertical green color values are 
averaged and horizontal and vertical blue color values 
induding a blue color site value provide a correction valuer 

7. The apparatus of daim 1 wherein, when the preferred 
orientation is horizontal, a color value is green, and the 
missing color value is red, adjacent hcaizontal red color 
values are averaged and horizontal green cdor values 
induding the green color value provide a correction value. 

8. The apparams of daim 1 wherein, when the preferred 
orientation is horizontal, a color value is green, and the 
missing color value is blue, adjacent horizontal blue color 
values are averaged and horizontal green color values 
induding the green color value provide a cwrection value. 
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9. The apparatus of dalm 1 wherdD, when the preferred 
orientation is negative diagonal, a color value is green, and 
the missing color value is red, adjacent negative diagonal red 
color values are averaged and negative diagonal green color 
values including the green color site value provide a cor- 
rection value. 

10. The apparatus of daim 1 wherein, when the preferred 
orientation is negative diagonal, a color value is green, and 
the missing color value is blue, adjacent negative diagonal 
blue color values are averaged and negative diagonal green 
color values including the green color site value provide a 
correction value. 

11. The ^aratus of claim 1 wherein, when the preferred 
orientation is vertical, a color value is red, and the missing 
color value is green, adjacent vertical green color values are 
averaged and vertical red color values including the red 
color site value provide a collection value. 

12. The apparatus of claim 1 wherein, when the preferred 
orientation is vertical, a color value is blue, and the missing 
color value is green, adjacent vertical green color values are 
av^^ed and vertical blue color values including the blue 
color site value provide a correction value. 

13. The apparatus of claim 1 wherein, when the preferred 
orientation is vertical a color value is green, and the missing 
color value is red, adjacent vertical red color values are 
averaged and veitical green color values including the green 
color site value provide a correction value. 



14. Hie ^aratus of claim 1 wherein, when the preferred 
orientation is vertical, a color value is green, and the missing 
cdar value is blue, adjacent vertical blue color values arc 
averaged and vertical green color values including the green 

5 color site value provide a correction value. 

15. The apparatus of claim 1 wherein, when the preferred 
orientation is positive diagonal, a color value is green, and 
the missing color value is red, adjacent positive diagonal red 
color values are av«:aged and positive diagonal green color 
values including the green color site value provide a cor- 
rection value, 

16. The apparatus of claim 1 wherein, when the preferred 
orientation is positive diagonal a color value is green, and 
the missing color value is blue, adjacent positive diagonal 
blue color values are averaged and positive diagonal green 

15 color values including the green color value provide a 
correction value. 

17. The apparatus of daim 4 wherein the predetermined 
value is 25 when red, green and blue pixel values arc 
represented by a digital range corresponding from 0 to 255. 

20 18. The apparatus of claim 5 wherein the predetermined 
value is 200 when red, green and blue pixel values are 
represented by a digital range corresponding from 0 to 255. 

19. The apparatus of claim 6 who'dn the predetermined 
value is 200 when red, green and blue pixd values are 

25 represented by a digital range corresponding from 0 to 255. 
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